1. Field of the Invention
The present invention relates to an information processing apparatus, eye open/closed degree determination method, computer-readable storage medium, and image sensing apparatus.
2. Description of the Related Art
A technique of detecting a face from an image including still and moving images has conventionally been known (Mitarai, Y., Mori, K., Matsugu, M. “Robust Face Detection System Based on Convolutional Neural Networks Using Selective Activation of Modules”, FIT (Forum of Information Technology), L1-013, 2003). Further, techniques have been proposed for determining the open/closed degree of eyes in a detected face.
In this regard, Japanese Patent Laid-Open No. 09-044685 proposes a technique of comparing the distance between the eyebrow and the pupil or the area ratio of the eyebrow and pupil with a predetermined reference, and calculating the change amount of the distance, thereby determining the eye open/closed degree.
Japanese Patent Laid-Open No. 11-242744 proposes a technique of grouping eye regions into a white region and black (pupil) region, and when the black region changes, determining that the object blinks.
Japanese Patent Laid-Open No. 2004-192551 proposes a technique of preparing a plurality of processes to determine the eye open/closed degree in order to distinguish an eye state immediately before closing eyes from an eye state upon a change of the expression such as a smile, and determining the eye open/closed degree on the basis of a result using the ANDs of respective determination results.
There is an individual difference in eye size (e.g., the distance between upper and lower eyelids or the appearance of the pupil) in an expressionless state while the eyes are fully open. For example, the pupil of person A shown in FIG. 19 is completely seen in an expressionless state (state 1) while the eyes are fully opened. In contrast, the pupil of person B shown in FIG. 19 is partially hidden in an expressionless state (state 4) while the eyes are fully opened.
In this manner, there is an individual difference in eye size in an expressionless state. If the eye open/closed degree is determined from only one image, state 2 of person A shown in FIG. 19 and state 4 of person B shown in FIG. 19 are determined to have the same eye open/closed degree. For example, when state 2 of person A shown in FIG. 19 is determined as a shooting failure upon shooting by an image sensing apparatus, state 4 of person B shown in FIG. 19 is also determined as a shooting failure.
When determining the eye open/closed degree using the amount of change between a plurality of images, the amount of change from state 1 to state 2 of person A shown in FIG. 19 equals that of change from state 4 to state 5 of person B shown in FIG. 19. It is determined that state 2 of person A and state 5 of person B are the same. For example, when state 2 of person A shown in FIG. 19 is determined as a shooting success upon shooting by an image sensing apparatus, state 5 of person B shown in FIG. 19 is also determined as a shooting success.
Determination of whether an image shot by an image sensing apparatus is a failure or success changes depending on the user of the image sensing apparatus. It is necessary to detect the eye open/closed degree at high precision regardless of individual differences in the eye size. For example, the above-mentioned technique disclosed in Japanese Patent Laid-Open No. 2004-192551 determines the eye open/closed degree by using a combination of eye open/closed degree determination results. However, each eye open/closed degree determination result is a binary output representing that the eyes are open or closed. For this reason, even this technique cannot detect the eye open/closed degree at high precision regardless of individual differences in eye size.
A case where the eye open/closed degree is determined by a method using one image, and a case where it is determined by a method using the amount of change between a plurality of images will be examined. A case where the eye open/closed degree is determined when it changes from state 1 to state 2 shown in FIG. 20, and a case where the eye open/closed degree is determined when it changes from state 4 to state 5 shown in FIG. 20 will be exemplified.
When the eyes change from state 1 (black eye area 10) to state 2 (black eye area 5) shown in FIG. 20, the change amount of the black eye area is 5, and the absolute amount of the black eye area in state 2 is 5. When an absolute amount threshold for determining that the eyes are closed is set to 6 (it is determined that the eyes are closed when the absolute amount is smaller than 6), and a change amount threshold for determining that the eyes are closed is set to 4 (it is determined that the eyes are closed when the change amount is larger than 4), both determination results based on the absolute amount and change amount represent that the eyes are closed. When the eyes change from state 4 (black eye area 5) to state 5 (black eye area 0) shown in FIG. 20 and the same determinations as those described above are made, the same results are obtained, and both determination results based on the absolute amount and change amount represent that the eyes are closed. In this manner, according to the conventionally proposed eye open/closed degree determination method, state 2 and state 5 shown in FIG. 20 are determined to have the same eye open/closed degree.